Method and apparatus for providing an infrared image

ABSTRACT

A method and a device to provide a full representation of an object having a wide dynamic intensive range and maintain a good sensitivity for signal levels throughout the whole intensive range is disclosed. The wide intensive range is divided into a predetermined number of intensive intervals (INT 1  to INT 3 ) inside the wide intensive range. The intensive intervals are provided at the side of each other or are partly overlapping. The same number of interval representations (IM 1  to IMn; F 1  to Fn; T 1  to Tn) of the object are provided as the predetermined number of intervals. Each interval representation is adapted to one individual of the intervals. The sizes, acquisition parameters and/or calibration parameters of the interval representations (IM 1  to IMn; F 1  to Fn; T 1  to Tn) are adapted to each other. The interval representations respective intervals are provided in the same full representation ( 18; 18′; 20 ).

[0001] This invention relates to a method and apparatus for providing aninfrared (IR) image. The invention relates especially to a method andapparatus for making an infrared image having a wide dynamic temperaturerange.

BACKGROUND

[0002] It has until now been difficult to create images particularly forinfrared cameras, which have a wide dynamic range, for example rangingfrom −5° C. to 1100° C., and to maintain a very good sensitivity acrossthe whole temperature range, and particularly for the lower signallevels.

[0003] It is a well known disadvantage in relation to pure IR imagesthat the users often have some difficulty to interpret the view that isshown. IR images are often more blurred than visual images because thewavelength region from the illustrated scene is different and thetransitions between different temperatures are rather smooth. It is tobe noted that an IR image is essentially based on different intensity ofthe shown objects within practically the same wavelength region, and theimage is thus not very dependent on the wavelength, unlike images basedon the visible wavelength region. Higher temperature at an object givesin principle a higher intensity. The different shades of intensity aregiven in a chosen colour scale, for example a high intensity is shown inred and a low brightness is shown in blue or violet.

[0004] A problem to solve is to make temperatures in a wide dynamictemperature range clear, distinct, and with good resolution even for thelower signal levels.

[0005] The invention has been particularly developed for use in IRcameras. However, it could also be used in any other kind of imagingdevice. One example of such a device is in infrared imagers for firefighters. In such an imager a line scanning is made from an air bornevehicle flying over the object, which can be a forest. A demand on suchsystems is that they shall be able to present images with goodresolution and accuracy both for room temperature and high temperaturescenes simultaneously.

DESCRIPTION OF RELATED ART

[0006] U.S. Pat. No. 5,249,241 describes a histogram projection system,which automatically optimises tracks changes in luminance and adjusts inreal time the display of wide dynamic range imagery from IR-cameras. Thetechnique described in this document assigns display dynamic rangeequally to each occupied intensity level in the raw data. Thus ahistogram projection system automatically tracks changes in luminance toadjust the display of wide dynamic range IR-imagery.

THE INVENTION

[0007] An object of the invention is to provide a method and anapparatus for providing an IR image, which presents a view, in which thetemperature differences are presented with good resolution all through ahuge temperature range.

[0008] Another object of the invention is to provide a method and anapparatus for presenting images with good resolution and accuracy bothfor room temperature and high temperature scenes simultaneously.

[0009] The objects mentioned above are solved with a method having thefeatures disclosed in the characterising part of claim 1. Furtherdevelopments and features and an apparatus to provide the method areapparent from the rest of the claims.

[0010] The invention relies on the fact that a time multiplexing andsequencing procedure acquires a sequence of images from a linear ormatrix array of radiation sensitive detector elements. This matrix arraycould be a Focal Plane Array (FPA), having radiation sensitive detectorelements, for example IR sensitive elements. The detector elements couldbe Micro-bolometers.

[0011] Advantages

[0012] The invention makes it possible to present images with goodresolution and accuracy by creating an image that is a mixture of atleast two images taken with different integration (exposure) times. Theinvention takes advantage on the fact that an Infrared Camera isradio-metrically calibrated and temperature stabilised, and also thatone can create an Image Pixel Stream with calibrated pixels. It is to benoted that the present invention could not be implement in anon-calibrated IR camera.

BRIEF DESCRIPTION OF THE DRAWINGS

[0013] For a more complete understanding of the present invention andfor further objects and advantages thereof, reference is now made to thefollowing description of examples of embodiments thereof—as shown in theaccompanying drawings, in which:

[0014]FIG. 1 illustrates the principle for a multi-range measurement;

[0015]FIG. 2 shows a block schedule of a multi-range solution accordingto a first embodiment of the invention;

[0016]FIG. 3 shows a block schedule of a multi-range solution accordingto a second embodiment of the invention; and

[0017]FIG. 4 shows a block schedule of a multi-range solution accordingto a third embodiment of the invention

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0018] Referring to FIG. 1, if the temperature range to be monitored foran object OBJ is very wide, for example ranges from ca −10° C. to ca1100°, an IR image from only one image taken on the object will have agood sensitivity only for signal levels obtained for the higher parts ofthe temperature range. However, it is often important to have a goodresolution also for signal levels from the lower part of the range.

[0019] According to the invention the wide temperature range is dividedinto at least two temperature intervals, three intervals are illustratedin FIG. 1. An image providing means, for example an IR camera, isprovided, which takes a succession of images, each related to anindividual one of the temperature intervals. As apparent from FIG. 1showing an image of an area, in which a forest fire has started, thearea to be observed has a temperature varying between ca −5° C. and ca1100° C. The variation is illustrated to be included in three mainareas, P1, having an area with a temperature between −5° C. and 150° C.,comprising an icy lake, copse, and a human body, P2, comprising areaswith a temperature between 100° C. and 600° C., comprising sparse flamesand heated soil, and P3, with a temperature between 500° C. and 1000°C., comprising burning and flaming trees and ground cover.

[0020] A camera circuitry or an acquisition board reconstructs a singleframe, shown in the upper part of FIG. 1, from the successive frames.The frames have preferably been cut such that no overlapping betweenthem exist before the reconstruction. There are at least two frames.More than three demands a long time, if the different images are takensuccessively. This can be too long if the camera should be tolerablyfast. Most often there is a demand to build up the successive frames inthe same rate as the ordinary video frequency. The object is not allowedto move or change its temperature fast between the image recordings.However, the scope of the invention is not limited to use of aparticular numbers of image recordings. However, as will be stated lateron there is a possibility to provide a fast camera based on use of anumber of FPAs.

[0021] In the embodiment shown in FIG. 1, an example of a full rangefrom −10° C. to 1100° C. is given, where a number of differentintegration times have been determined. Three different images are thenavailable with three different temperature intervals, the first one INT1from −10° C. to 100° C., the second one INT2 from 90° C. to 600° C., andthe third one INT3 from 500° C. to 1100° C. These three temperatureintervals are only mentioned as exemplary and illustrated under theobject OBJ. This will give three images with different useful areas,basically corresponding with each individual of the parts P1, P2, P3.

[0022] In the lower part of FIG. 1 the three intervals are illustratedin relation to the wide temperature range R ranging from −10° C. to1100° C. and to illustrate the interface between the differenttemperature intervals when they are shown in a common display. Thedifferent temperature intervals are preferably provided with anindividual colour scale each in order to distinguish the temperatureinterval from each other.

[0023] The goal is then to reconstruct from the consecutive frames acomposed image comprising only a “useful” frame, which has the fulldynamic range from −10° C. to 1100° C. The frame speed is adjusted withthe number of successive frames with different integration times suchthat the “useful” frame has a constant frame rate, for example 50 Hz (20ms). Thus, preferably, the intention is to produce each useful frame inreal time without any processing later on.

[0024] Referring to FIG. 2, a linear or matrix array 1 of radiationsensitive detector elements, such as a focal plane array (FPA)preferably comprising micro-bolometers. However, other kinds ofradiation sensitive detector elements could be used as-well. As commonfor IR cameras using FPA it is radio-metrically calibrated andtemperature stabilised.

[0025] Several images IM1, IM2, . . . IMn, representing intervalrepresentations since they represent intensive intervals, are to betaken in this embodiment for different temperature regions, i.e. thewide temperature range to be dynamically presented in an image isdivided into an appropriate number of minor temperature intervals. Theseminor temperature intervals could preferably be chosen to be alike, butthey could also be chosen such that the lowest temperature interval isthe smallest and that the rest of the minor temperature intervalsembrace larger and larger temperature regions. It is to be noted thatother kinds of intensive intervals could be embraced by the inventiveidea, such as intensive changes within the visible wavelength region.

[0026] For example, the successive temperature intervals could haveincreased intervals along an exponential scale. It is to be noted thatthe different images for the different temperature regions are recordedin mutually different integration times, the longer the warmer region.These different integration times must be compensated for when buildingup the useful frame.

[0027] Coarse maps 2A to 2 n, one for each temperature region, forexample each having 240*320 values, provide values used for coarseoffsetting of the individual pixel values for each successive recordingof the FPA 1 before the sensed values of the FPA are analogue-to-digitalconverted in an A/D converter 3. It is to be noted that the A/Dconverter need not be a separate element as illustrated. It could beincorporated in the FPA itself.

[0028] Analogue Global Offset (AGO) fields 4, 5, 6, one for each imageto be taken, each representing an individual temperature region, areconnected to the FPA 1 in order to impose AGO values on each pixelsignal for an image just being recorded. This is also provided beforethe values from the FPA 1 are analogue-digital converted.

[0029] There is a set of acquisition and calibration parameters providedfor each image IM1, IM2, . . . IMn.

[0030] At least as many Offset map fields 7, 8, 9 of digital values asthere are images taken are used to individually offset each pixel valuein each image separately. In the Offset map arrays shown in theembodiment of FIG. 2 each pixel value will be compensated for the CameraInternal Temperature Drift (CITD).

[0031] The digital values from the FPA, via the A/D converter 3, and thevalues from the Offset map fields 7, 8, 9 are summarised for each imageIM1, IM2, . . . IMn separately in a summarising means 10. In this wayeach pixel in each image is offset compensated including offset driftdue to CITD.

[0032] One Gain map & bad pixel replacement array 11, 12, 13 per imageIM1, IM2, . . . IMn is provided for individually gain compensate eachpixel value. Each value in each of these arrays is adjusted for thecamera internal gain drift due to camera temperature variations.

[0033] A multiplier 14 makes a multiplication image by image with theGain map & bad pixel replacement array 11, 12, 13. In this way eachpixel is gain compensated including gain drift due to the CITD.

[0034] Thus, as by the measures discussed above, the acquisitionparameters, which are a set of scalar and matrix variables, will be usedto control and correct the acquired image. As an example, this could beOffset and Gain correction maps used in an IR camera. The calibrationparameters is a set of variables obtained during a calibration procedurewhen the camera is manufactured and imposed on each image IM1, IM2, . .. IMn. Due to this calibration procedure each image IM1, IM2, . . . IMnwill be mapped into an object temperature range, i.e. the image IM1 willhave a temperature range Ta to Tb, the image IM2 a temperature range Tbto Tc, . . . , and the image IMn a temperature range Tn−1 to Tn.Ta<Tb<Tc<<Tn−1<Tn.

[0035] Thus, there will be a successive stream of images taken on thesame object. A new, combined image, below called Display Image, couldthereafter be built. Pixel values in the Display Image are thus takenfrom the successive images combined with each other in a selectionprocedure, of which an embodiment is described below.

[0036] Selection Procedure

[0037] Pixel values related to temperatures Ta to Tb are chosen from theimage IM1, pixel values related to temperatures Tb to Tc are chosen fromthe image IM2, etc. Naturally the amount of images are depending uponhow many different types of images and temperature intervals one wantsto map into the Display Image. It is to be noted that it could bepossible to change the temperature interval for each image and thus havemore or less images for different kinds of applications for the camera.

[0038] This selection procedure can be implemented by using either oneLook Up Table (LUT), which is updated synchronously to the timemultiplexed behaviour of the image acquisition, or a set of LUTs, onefor each image IM1, IM2, . . . IMn. The LUT or LUTs are used to linearlyor non-linearly map pixels in the separate images IM1, IM2, . . . IMninto the Display Image in accordance with the selection procedure.

[0039] The embodiment shown in FIG. 2 comprises one LUT 15 used for thepresentation. The LUT need not normally be temperature compensated.Depending upon what integration interval the LUT 15 is to be working init comprises a different transmission function to transmit the signalinto the display after being buffered in a temperature image storage 18.

[0040] The outputs from each LUT are thus combined in the digitaltemperature image storage 18, which preferably provides an 8-bittemperature image with as many ranges as there are images taken. Eachimage is for example 240*320 pixels. The digitised output from thestorage 18 is then sent as a data stream for further processing, forexample in a video digital/analogue converter, and from there to beshown on a display (not shown) to show the combined Display Image.

[0041] The LUT is updated as a part of a temperature compensation. Thus,the LUT need adjustment as a function of what temperature region itactually is working on.

[0042] Functional Description of LUT (Look up Table)

[0043] For simplicity reasons, the description below will be given for atwo range solution.

[0044] Y=Display memory depth

[0045] For two range LUTs two temperature intervals will be mapped intoone image. Then, three object temperatures are needed, where at leastthe middle temperature is common for the two temperature intervals. Itis to be noted that the chosen temperature intervals could overlap eachother to some extend at the interface between them.

Ta_(OBJ)<Tb_(OBJ)<Tc_(OBJ)

[0046] where Ta_(OBJ) and Tc_(OBJ) are the lowest and the highesttemperatures of the object to be monitored within the wide dynamictemperature range, and Tc_(OBJ) is a middle temperature at the interfacebetween the two temperature intervals that the wide temperature range isdivided in.

[0047] A transfer function between pixel values and object temperaturefor the image IM1 acquisition and calibration parameters (temperatureinterval 1) is:

U _(PIX) =f ₁(T _(OBJ)))

T _(OBJ) =f ₁ ⁻¹(U _(PIX))

[0048] A transfer function between pixel values and object temperaturefor the image IM2 acquisition and calibration parameters (temperatureinterval 2) is:

U _(PIX) =f ₂(T _(OBJ)))

T _(OBJ) =f ₂ ⁻¹(U _(PIX))

[0049] The LUT for the temperature interval 1, LUT1, maps pixel valuesinto the display memory in the following manner:

for U _(PIX) =f ₁(T _(OBJ)) <Ta _(OBJ))

LUT 1 pix=0

for U _(PIX) =f ₁(Ta _(OBJ) ≦T _(OBJ) <Ta _(OBJ))

LUT 1 pix=0 to X

[0050] where X is a value between 0 and the depth of the display memory.For an 8-bit display memory X could be between 0 and 255.

for U _(PIX) =f ₁(Tb _(OBJ) ≦T _(OBJ))

LUT 1 pix=0

[0051] Thus these pixels will be over read by LUT2.

[0052] The LUT for the temperature interval 2, LUT2, maps pixel valuesinto the display memory in the following manner:

for U _(PIX) =f ₂(T _(OBJ)) <Tb _(OBJ))

LUT 2 pix=0

[0053] These pixels will be over read by LUT1

for U _(PIX) =f ₂(Tb _(OBJ) ≦T _(OBJ) <Tc _(OBJ))

LUT 2 pix=X to Y

[0054] where X is a value between 0 and the depth of the display memory.For an 8-bit display memory X could be between 0 and 255 and Y=255.

for U _(PIX) =f ₂(Tc _(OBJ) ≦T _(OBJ))

LUT 2 pix=Y or 0

[0055] However, in order to provide a fast camera it is possible toprovide a number of focal plane arrays (FPAs), or other kind orrecording features, one for each temperature interval, instead of onlyone FPA. An embodiment illustrating this feature is shown in FIG. 3. Inthis embodiment each FPA F1 to Fn has a circuitry of its own up to a LUTof its own. The difference between this embodiment and the embodimentshown in FIG. 2 is that the processing procedure of the recorded imagesis parallel in FIG. 3 and successive in FIG. 2. The different blocksshown in FIG. 3 having the same tasks as the ones shown in FIG. 2 areprovided with the same references except for an “′”. The separatesummarising means for the three images have the references 10A′, 10B′,and 10C′, respectively. The separate multiplier means for the threeimages have the references 14A′, 14B′, and 14C′, respectively.

[0056] The embodiment shown in FIG. 3 there is one LUT 15A′, 15B′, and15C′ for each image F1, F2, . . . Fn. The frame speed of the LUTs 15A′,15B′, 15C′ are adjusted in a combination and adaptation circuit 16 toprovide the full representation, i.e. the temp image, with the number ofsuccessive interval representations, i.e. fields, having differentintegration times for the different interval representations, such thatthe display rate for the combined interval representations has apredetermined frame rate.

[0057] Thereafter, the images are provided as a combined and adaptedtemperature image information in a digital temperature image storage18′, which preferably provides an 8 bit temperature image with as manyranges as there are images taken. Each image from the LUTs is forexample 240*320 pixels.

[0058] The digitised output from the storage 18′ is then sent to a videodigital/analogue converter and from there to be shown on a display (notshown) to show the combined Display Image. The sensitivity will be atleast slightly different for the three sensor-groups (FPAs and lineelements). A compensation must be provided. Then the sensitivity of theimage as a whole will be inferior that of an image recorded successivelyon the same sensor group. This depends upon that the bandwidth isincreased and it is related to the sensitivity.

[0059] Referring to FIG. 4, for an imager, which comprises a linearsensor element array, there could be one line, or a few parallel lines,of sensor elements for each temperature region, and they could besuccessively imagined, at the line recordings, such as in the embodimentshown in FIG. 2. The same kind of elements as in FIG. 2 could be usedand have thus the same references, even though they have a diminishedsize compared to those shown in FIG. 2. However, the “image” for eachrecording is only one line, or a few lines, high, which means that it ispossible to have more successive recordings for the same display line orlines, since they take shorter time. In this embodiment one LUT 22, 23,and 24 is provided for each line T1, T2, and Tn, respectively. Thedisplay image 25 is then built up line by line in a scrolling manner.

[0060] This embodiment could for example be used when flying over anarea, in which a forest fire is to be monitored, such as the one shownin FIG. 1, or in process monitoring to make a thermal map ofelectrolytic tanks to show short circuits, defective electrolyte flows,hot current rails and/or missing anodes. Here an imager could betransported over an area comprising the electrolytic tanks.

[0061] Even in such a multi-line sensor device the lines are to becombined into one to be presented on a single Display in a scrollingprocedure. It is also possible to provide a system of parallelrecordings of line representations like the way illustrated in FIG. 3.

[0062] Although the invention is described with respect to exemplaryembodiments it should be understood that modifications can be madewithout departing from the scope thereof. Accordingly, the inventionshould not be considered to be limited to the described embodiments, butdefined only by the following claims, which are intended to embrace allequivalents thereof.

1. A method to provide a full representation of an object having a widedynamic intensive range and maintain a good sensitivity for signallevels throughout the whole intensive range, characterized by thefollowing steps: a) divide the wide intensive range into a predeterminednumber of intensive intervals (INT1 to INT3) inside the wide intensiverange, which intensive intervals are provided at the side of each otheror are partly overlapping; b) providing the same number of intervalrepresentations (IM1 to IMn; F1 to Fn; T1 to Tn) of the object as thepredetermined number of intervals, each interval representation beingadapted to one individual of the intervals; c) adapting the sizes,acquisition parameters and/or calibration parameters of the intervalrepresentations (IM1 to IMn; F1 to Fn; T1 to Tn) to each other; d)combining the interval representations such that their respectiveintervals are provided in the same full representation (18;18′; 20). 2.A method according to claim 1, characterized by providing thepredetermined intervals to be non-overlapping before combining theinterval representations into the full representation.
 3. A methodaccording to claim 1, characterized by providing the intervalrepresentations for the predetermined number of intervals successivelyin a cyclic way.
 4. A method according to claim 3, characterized byadjusting (18; 25) the frame speed of the full representation with thenumber of successive interval representations in a cycle with differentintegration times for the successive interval representations, such thatthe display rate for the combined interval representations has apredetermined frame rate.
 5. A method according to claim 1,characterized by adjusting (16,18′) the frame speed of the fullrepresentation with a number of parallel recorded intervalrepresentations for a full representation with different integrationtimes for the individual interval representations, such that the displayrate for the combined interval representations has a predetermined framerate.
 6. A method according to claim 1 for providing infrared images,characterized by making aradiometric calibration of each recordedinterval representation before combining of a cycle of intervalrepresentations.
 7. A method according to claim 1, characterized bytemperature stabilisation of each recorded interval representationbefore combining a cycle of interval representation.
 8. A methodaccording to claim 6, characterized by creating for each intervalrepresentation a pixel stream having calibrated pixels.
 9. A methodaccording to claim 1, characterized by providing the intervalrepresentations by means of a linear or matrix array of radiationsensitive detector elements, for example comprising Focal PlaneArrays(FPAs) of Micro-bolometers.
 10. A method according to claim 1,characterized by providing a Look Up Table (LUT) common for all theinterval representations in a cycle; updating the Look Up Tablesynchronously to a time multiplexed behaviour of the intervalrepresentation acquisition.
 11. A method according to claim 1,characterized by providing a set of Look Up Tables (LUT), one for eachinterval representation in a cycle.
 12. A device for an intensive fullrepresentation creating means for providing representations within awide intensive range, comprising at least one linear or matrix array ofradiation sensitive detectors for recording interval representations,characterized by means (1; F1 to Fn; 21) for recording a predeterminedinterval representation, each recorded for an individual intensiveinterval within the wide intensive range; means (7 to 15; 7′ to 15A′,15B′, 15C′; 7 to 24) for discriminating the intensive intervals suchthat the individual intensities for the individual intervalrepresentations will lie at the side of each other; combination means(18; 16,18′; 25) for combining the individual interval representationsto a common full representation.
 13. A device according to claim 12,characterized by successive means (2 to 15; 2 to 24) providing theinterval representations for the predetermined number of intervalssuccessively in a cyclic way.
 14. A device according to claim 13,characterized by combination and adaptation means (18; 25) for the framespeed of the full representation with a number of successive intervalrepresentations in a cycle with different integration times for thesuccessive interval representations, such that the display rate for thecombined interval representations has a predetermined frame rate.
 15. Adevice according to claim 12, characterized by combination andadaptation means (16,18′) the frame speed of the full representationwith a number of parallel recorded interval representations for a fullrepresentation with different integration times for the individualinterval representations, such that the display rate for the combinedinterval representations has a predetermined frame rate.
 16. A deviceaccording to claim 12 for providing infrared lines or images,characterized by radiometric calibration means comprised in the means(1; F1 to Fn; 21) for recording a predetermined interval representation.17. A device according to claim 12, characterized by temperaturestabilisation means (7,8,9; 7′,8′, 9) of each recorded intervalrepresentation before combining a cycle of interval representation. 18.A device according to claim 16, characterized by gain compensation means(11, 12,13; 11′, 12′, 13′) for creating for each interval representationa pixel stream having calibrated pixels.
 19. A device according to claim12, characterized by a linear or matrix array of radiation sensitivedetector elements, for example comprising Focal Plane Arrays (FPAs) ofMicro-bolometers, for recording the interval representations.
 20. Adevice according to claim 12, characterized by a Look Up Table (LUT)common for all the interval representations in a cycle updatedsynchronously to a time multiplexed behaviour of the intervalrepresentation acquisition.
 21. A device according to claim 12,characterized by a set of Look Up Tables (LUT), one for each intervalrepresentation in a cycle.